Information Extraction with and without Parsing Semi-structured Documents

Information extraction from semi-structured documents comprises contents detection, wrapper generation and schema extraction. The contents detection step corresponds to making training examples in wrapper induction based on machine learning and the schema extraction identifies extracted data types. We formulate the contents detection using the repetitive pattern introduced in this paper. That is, we define the contents detection problem which is, given strings generated by some repetitive pattern, to find constant strings of the pattern. Then, we develop a linear time algorithm for the problem. This implies that it is not necessary for the algorithm to parse given documents except for a saw which is a set of characters pointing the boundaries of the contents and the other parts of documents. The necessity of the saw is also supported by experiments. The parsing, however, is shown to play an important role in the schema extraction step.